A Learnability Model for Universal Representations and Its Application to Top-down Induction of Decision Trees

Automated inductive learning is a vital part of machine intelligence and the design of intelligent agents. A useful formalization of inductive learning is the model of PAC-learnability. Nevertheless, the ability to learn every target concept expressible in a given representation, as required in the PAC-learnability model, is highly demanding and leads to many negative results for interesting concept classes. A new model of learn-ability, called Universal Learnability or U-learnability, recently has been proposed as a less demanding, average-case variant of PAC-learnability. This paper uses the U-learnability model to analyze a top-down decision tree induction algorithm. Speciically, this paper proves that an idealized variant of the well-known decision tree learning algorithm CART|one of the most successful existing machine learning algorithms|is a U-learner under a natural set of assumptions regarding target hypotheses. (The motivation and description of these assumptions is best delayed until the U-learnability model is described.) Equally interestingly, various related PAC-learning algorithms such as those for k-DNF cannot be used to U-learn under the same assumptions. Finally, the paper raises a number of 1 related open questions and general research directions; open questions include not only U-learnability questions, but also several new PAC-learnability questions and one question regarding a general property of propositional logic.